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Abstract 
CN 

H 

There is evidence that flying animals such as pigeons, goshawks, and bats use optical flow sensing 



to enable high-speed flight through forest clutter. This paper discusses the elements of a theory of 
controlled flight through obstacle fields in which motion control laws are based on optical flow sensing. 
Performance comparison is made with feedback laws that use distance and bearing measurements, 
^H and practical challenges of implementation on an actual robotic air vehicle are described. The related 

. question of fundamental performance limits due to clutter density is addressed. 
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1. INTRODUCTON 
CO 

O 

In the natural world, one finds enormous diversity in visual capabilities among different species 



of animals. Birds of prey typically possess significantly greater visual acuity than humans (or 



other animals), and while most mammals are inferior to humans in visual acuity, they have 
superior abilities in detecting motion. In the present paper, possible mechanisms by which 
certain flying animals use visual sensory feedback to control their movement though cluttered 
environments are discussed. The principal emphasis will be on the use of optical flow techniques. 
Although there is growing evidence that flying animals use optical flow sensing to navigate 
and control their motions (0), the ways in which such sensing is integrated with other sensory 
modalities (inertial sensing, stereo visual depth perception, etc.) is not well understood. Flying 
animals tend to have constrained visual capabilities — e.g. immobile eyes with fixed-focus optics. 
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These animals must therefore employ visual strategies that are adapted to their specialized visual 
abilities. Many of these strategies emphasize visual cues derived from motions of retinal images. 

(See ma.) 

The interplay between motion control and motion perception has recently been studied by 
a number of researchers. Some of this work has been inspired by animal predators that have 
developed the ability to camouflage their motions when approaching to capture prey. The key 
to this strategy is to control the apparent relative motion as perceived on the retina of the prey. 
dfT3lO In the spirit of the research reported below, Justh and Krishnaprasad ([9]) have proposed 
a biologically plausible feedback flight control law that exhibits motion camouflage relative to 
a moving target. 

The present paper describes some of our recent work aimed at developing robotic motion 
control control laws that produce motions resembling those of animals. Of particular interest is 
high-speed flight through forest clutter. There are a number of fundamental research questions 
that the work addresses. While different flying animals employ complex combinations of sensing 
modalities, a fundamental premise of the paper is that in idealized environments, it should be 
possible to successfully use optical feedback alone to guide a UAV through an obstacle field. 
The research has the following components: 1. We are continuing the development of operating 
regime dependent motion primitives ([2]) that will create a set of behaviors that is rich enough 
to achieve our goal of high-speed flight through a field of obstacles. 2. We would like to use 
optical sensory feedback in a way that reflects what is known about the mechanisms of animal 
flight and in a way that the motions that are generated by the proposed laws have qualitative and 
quantitative characteristics of animal motions observed in nature. 3. Finally, we would like to 
understand implementation independent fundamental bounds on the performance limits of flight 
through models of forest and other types of clutter. Recent work by Karaman and Frazzoli [8] has 
shown that under a certain idealization there is a critical velocity below which a mathematical 
model of a bird can fly through a model forest without colliding with a tree yet above this critical 
velocity, an infinitely long collision-free flight path does not exist. Our goal is to understand 
how the answers to such questions change as we vary our optics -enabled control designs. 
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A. Interpreting optical flow generated by thin objects with sharp boundaries 

Perhaps the single-most important feature of optical flow sensing is that when a vehicle is 
moving toward an object, it enables a very simple determination of time-to-contact under constant 
closing velocity. This determination can be made without knowledge of the size, distance, or 
velocity of motion toward the object. To see this transparently, we consider the idealized thin 
obstacle depicted in Fig. [1] If the obstacle is observed using a pinhole camera as depicted, it 
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Fig. 1. A thin object of diameter d is approached at a constant velocity x(t) = v. We assume the obstacle registers on the 
image plane of a pinhole camera, and the length of the image is di. 

creates an image on the focal plane of the camera. The variables here depicted are d = diameter 
of the object, di = diameter of the image on the image plane, x = distance from the camera to 
the object, and / = the focal length of the camera. These variables are related by the fundamental 
constitutive equation 

d - = % (i.i) 

x j 
Suppose the camera moves toward the object at constant speed, x(t) = —v with v > 0. Denote 
x(0) by Xq. As x(t) decreases, di = di(t) increases. The quantities f,d remain fixed. Using the 
constitutive equation ( |1.1| ), it is possible to derive a differential equation governing the evolution 
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of the image size di 



diit) = -^-Mtf. (1.2) 



When v is held to any constant positive value, it is not surprising that there is a finite escape 
time for ( |1.1[ ). The easily derived solution 

dt(t) = -^- 

Xq — V ■ t 

shows this to be r = Xq/v. Remarkably, if it is assumed that none of the quantities d, xq, or v 
is known, it is still possible to determine r. Rewriting (TTTJ) as 



d- f = di(t) ■ x(t), 

and differentiating both sides, we have di(t) x(t) —vdi(t) = 0. Under the assumption of constant 
closing velocity, this may be written as 

di Xq — V t 

d\ v 

This is zero when t = Xq/v (the time of contact), and at t = 0, we see that di/di = Xq/v = r 
is the time remaining until contact. 

While this analysis has been done in terms of the observed changes in an image of an ideally 
situated idealized object, it in fact applies more generally to optical flow associated with any 
feature point in the field of view. The general conclusion is that if di(t) is the location of an 
image feature in the image plane, r = diit) /diit) is the time remaining until the camera is 
directly abeam of the actual feature. This will be discussed in greater detail in Section [3] 

The time-to-contact parameter that we have called r has been the focus of research in the 
psychology of human and animal perception. Kaiser and Mowafy (flU) have noted that concepts 
closely related to time-to-contact are relevant in a variety of settings in which motion perception 
is important. In particular, they have reported experiments in which humans are required to use 
optical flow information from screen images to estimate the time remaining before a simulated 
object is passed. The work indicates that subjects have greater difficulty estimating time-to-pass 
for objects that are significantly off the axis of approach. Lee and Reddish (0T||) present evidence 
that t plays a role in the way that gannets time their dives into the ocean in attempts to catch 
fish, and Wang and Frost (" lfT5lO have reported finding a part of the brains of pigeons that respond 
selectively to approaching objects in a way that is consistent with r being the key perceptual 
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variable. Like the finding of Kaiser and Mofawy, the response in the pigeon brain was markedly 
reduced when the object deviated 5% or more from a collision course. 

The contribution of the present paper is to develop an understanding of r in cases where the 
optical sensor is moving at a nonconstant velocity and to use this understanding to design motion 
control algorithms based on sets of r's associated with selected features in the sensor image. 
We are not aware of previous attempts to do this, and it is hoped that the kinds of designs under 
development will be useful in creating animal-like controlled motions of UAV's. In Section 3 
below, we introduce the concept of time-to-transit with respect to a feature point, which we 
denote by t(£), the time-dependency indicating that r may vary according to how the optical 
sensor accelerates or follows a curved path with respect to the feature point. 

The organization of the paper is as follows. Section II proposes a quantized steering model of a 
Dubins-like model of planar nonholonomic motion. Inspired by work by Karaman and Frazzoli 
([8]), the controlled motions of quantized Dubins vehicles through what we term Markovian 
obstacle fields are analyzed. It is shown that the likelihood of a controlled motion successfully 
transiting an obstacle field without collision depends on both the size and density of obstacles in 
the obstacle field and the steering authority (effectively the turning radius) that can be applied by 
the vehicle's controller. For a wide range of obstacle densities, it is shown that there are critical 
levels of steering authority, slightly below which it is almost impossible to transit an obstacle 
field and slightly above which it is almost certain the there will be a realizable collision-free path. 
In Section III, we consider motion control based on optical sensing and introduce the concept 
of time-to-transit a feature point. Control laws based on sensing of proximity and heading are 
discussed along with a control law based on times to transit certain key feature points. Section 
IV describes an implementation of optical flow algorithm on an actual quadrotor UAV using a 
GoPro camera. Conclusions and plans for further research are given in Section V. 

2. A Markovian Obstacle Field 

Consider an infinite line of one-dimensional obstacle slats as depicted in Fig. [2} The lengths 
and positions of each slat are random variables whose lengths and placement a modeled by a 
Markov law as follows. Let s denote position along the infinite line. The line is partitioned into 
a countable number of adjacent open and closed segments. We denote the defining partition as 
{■ • • < S-i < so < s\ < . . .} which is alternately composed of obstacle slats (closed segments 
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of the form [s 2 fc, Ssfc+i]) and open spaces (open segments (s 2 k~i, s 2 k)). The position s lies in 
an open space with probability pi(s) and on an obstacle slat with probability p( 2 (s). Note that 

Pl(s) +P2(s) = 1. 

A point s may or may not lie on an obstacle slat, and in either case, as we consider points to 
the right of s, a slat boundary will be encountered sooner or later. As the value of s increases 
the dependence of our probabilities on the spatial variable is 

Pi(s + As) = (1 - (3 As) Pl (s) + a As p 2 (s) + o(As) 
p 2 (s + As) = (3Asp 1 (s) + (l-aAs)p 2 (s) + o(As), 
where as s increases, the probability of coming to the left edge of a slat in the next As inches, 
given that the point s lies in open space, is (3 As + o(As), while the probability of coming to 
the right edge of a slat, given that s lies on a slat, is a As + o(As). In the limit, as As — > 0, 
these equations may be rendered 

dp l /ds = -pp^ + ap^s), 
dp 2 /ds = (3pi{s) - ap 2 (s). 

We assume that the distribution of slats is spatially stationary, and it is immediate that the 
stationary distribution is p\ = a/(a + (3), p 2 = (3/(a + (3). Thus, for any point s, the probability 
of s lying on a slat is p 2 , and the probability that it lies in open space is p\. 

We can describe that statistical characteristics of the sizes of obstacles and open spaces as 
follows. Suppose the point s = lies in open space between obstacles. 

Set up in this way, the distribution governing the occurrence of an obstacle edge as the variable 
s increases in an interval of open space is exponential with parameter (3. This means that the 
mean width of the open space is 1/(3, while the variance is l/(3 2 . Similarly, the right hand 
obstacle edge is exponentially distributed with parameter a, and the mean obstacle width and 
variance are 1/a and 1/a 2 respectively. Assuming the open space is on average wider than the 
obstacles (1/(3 > 1/a), the variance in the size of the open spaces is larger than the variance in 
obstacle widths as well. 

This characterization of one-dimensional arrays of obstacle slats provides a basic building 
block for constructing an idealized model of a two-dimensional obstacle field. In Fig. |2} rows 
of one-dimensional obstacle arrays are stacked vertically. To represent exponential randomness 
in the y-direction, we assume that each obstacle is displaced from a mean row position by an 
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World coordinates 



Fig. 2. A Markovian obstacle field. Obstacle heights are one pixerl while widths follow an exponential distribution with "rate" 
parameter a = 1.0. The between-the-obstacle open spaces follow exponential distributions with rate parameters /3 = 0.1, 7 = 0.1 
in the x and y directions respectively. With these parameters, there is clearly a reasonable likelihood that straight flight through 
the obstacle field as illustrated will not be impeded. 



amount that is exponentially distributed with parameter 7. In Figure [2| the average obstacle 
width has parameter 1/a where a = 1.0, while the average open space in both the horizontal 
and vertical directions have length parameters 1/(3 = 1/7 = 10. 

Having thus constructed a two-dimensional Markovian obstacle field, it is easy to compute the 
likelihood of a point vehicle being able to transit its depth (in the y-direction) without colliding 
with an obstacle. A straight collision-free path is depicted by the blue arrow. Assuming that 
the obstacles may be grouped into horizontal rows in each of which the obstacle widths and 



separations are given by the stationary versions of (2.3) (i.e. p\ = a/(a + (3), p 2 = (3 /(a + /?)), 
we find that the probability of transiting exactly n rows before encountering an obstacle is 

P„ 



PlP2- 



From this, one can calculate the probability of unobstructed motion through at least n rows of 
obstacles: 



n— 1 

Q n = l- S TP, 



k=0 



k = Pi 



Finally, the mean length of a collision-free path entering the obstacle field from a randomly 
chosen point along the horizontal axis is 



with the corresponding variance being 
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That the standard deviation is always larger than the mean indicates that uncontrolled motion 
through even a sparse obstacle field is risky. For the parameter values of Figure [2j the mean 
unobstructed path is ten length units long, while the variance is 1 10 (~ standard deviation 10.49). 

Continuing the abstraction of the obstacle field as rows of one-dimensional slat obstacles, 
we come to the question of the probability of constructing a collision-free path by a point 
vehicle through an obstacle field that contains a specified number of rows. It is assumed that the 
point vehicle in question executes Dubins-type motions — the vehicle's orientation is specified 
by a parameter 9, its forward velocity (x, y) has unit magnitude and is constrained to satisfy 
y cos 9 — x sin 9 = 0, and its turning radius cannot be less than some minimum R cr . Under these 
assumptions, the point vehicle may not be able to avoid colliding with a large obstacle lying 
directly in its path. 

To connect the kinematics of a steerable planar point vehicle with the above probabilistic 
analysis, we introduce a quantized idealization of Dubins vehicle motion. Specifically, we assume 
the vehicle always moves along a straight line path but can instantaneously change direction by 
an amount A9 such that < |A0| < 9 cr . 

In the quantized Dubins vehicle kinematics, a circular arc segment is replaced by an instan- 
taneous stright line path that has the same beginning and ending points as the arc. This straight 
line segment is the chord of the arc between the given endpoints. The quantized Dubins vehicle 
is steered though the obstacle field by means of the following control protocol. Note that all 
path segments are straight lines. Note also that the protocol admits the possibility of collision 
with an obstacle. The probability of such a collision is given by the proposition that follows. 

Proposition 2.1. Consider an n-row obstacle field in which the obstacles in each row lie along 
a line with the k-th row being displaced from the k— 1-st row by an amount I/7. The probability 
that there exists a collision free path through such an n-row obstacle field is 

(Pi + P2(l - e - (a/7) Wer )) B . (2.4) 



Remark 1. For the parameter values of Fig. [2] the dependence of this probability on 9 cr is 
shown in the next figure. Even modest steering capability dramatically increases the likelihood 
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Quantized Dubins Control Protocol 



1 



Approach the obstacle field heading In a direction 
perpendicular to the first obstacle row 



Is there an obstacle blocking the current path? 



NO 



Proceed to next obsta cle 
row. | Return to T7| 



NO 



I Crash 1 1 



YES 



Can the obstacle 
be avoided by A 8 
change In heading? 



YES 



Change heading by A 9. 



I Proceed to next obstacle row I 



Change heading by - A 6 

to realign the vehicle heading 

to be transverse to the 

obstacle row 



I Return to J71 



of existence of a collision-free path. Moreover, for each number n of obstacle rows, there is a 
value of steering angle 6* r above which it is virtually certain that a collision free path exists. 

Remark 2. Although the details of the curve shapes differ as the parameters a and (3 vary, 



the form of the dependence of (2.4) on 9 cr does not change qualitatively. Thus, even for dense 
arrays of thick obstacles, collision-free paths are possible — provided there is sufficient steering 
authority. 

Remark 3. The proposition will be proved under the assumption that the obstacles lie on 
horizontal lines as opposed to having exponentially distributed probabilistic separation in the 
"?/-axis" direction, which is the case depicted in Fig. [2} This uniformity assumption can be 
relaxed, and we expect that the probability of existence of obstacle avoiding paths will have a 
qualitatively similar expression. 



Proof: (Of Proposition |2.1[ ) There are exactly two ways in which the point vehicle 



can 
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Maximal change in heading (radians) Q 



cr 



Fig. 3. The probabilities of collision-free paths through n evenly spaced rows of obstacles as functions of Markovian parameters 
a, ft, depth parameter 1/7, and critical steering angle 6 cr . 



avoid colliding as it approaches a row of obstacles. It may be the case that it approaches the 
obstacle row at a point lying in open space between obstacles. The probability of that is pi. It 
may also be headed for a collision with an obstacle (the probability of which is p 2 ) and yet steer 
left or right by an amount ±A6> where A9 < 9 CT . using the steering protocol described above. 
The probability that there is enough steering authority to get past the left or right edge of the 
obstacle is i _ e -( a /7)tane cr since being able to steer past an obstacle by changing heading by an 
amount less than 9 cr is independent of whether an obstacle is encountered, the joint probability 
of encountering and obstacle AND being able to avoid it is ^(1 — e~^ a / 7 ^ tan6,c ' ). Thus, the 
probability of being able to avoid a collision in any row is 

Pi+p 2 (l-e- (a/7)tanec '-)- 

Since the rows have i.i.d. probabilities of not colliding, the probability of a collision-free path 
through n rows of obstacles is given by ( |2.1[ ) as claimed. ■ 



3. Motion control using optical sensor feedback 

In order to prescribe a steering algorithm based on optical flow servoing, we revisit the simple 
model of time-to-contact introduced in Section [TJ The goal will be to develop motion control 
laws that use images and sensed optical flow to automatically avoid obstacles. For simplicity, 
only planar motion is considered. While there is no essential difficulty in extending the treatment 
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to the case of motion in three dimensions, the restriction to 2-D does not appear to be overly 
restrictive in trying to understand animal flight through forests since bats and birds tend to 
pick flight levels above or immediately below canopies of branches and foliage where the only 
obstacles are the trunks of trees. (See e.g. [6].) We extend the analysis of Section[T]by considering 
a more general class of motions of a mobile optical sensor as depicted in Figure |4j 



o 
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Oi,-y(t) Xr {t)' 
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s X£(t) 



mobile ".''" "I^3" '\' 

optical ;' / / 



sensor •, < » ■ = — ? 
•..d r d(.- 



Fig. 4. Thin objects and the open space between are approached at a constant velocity x(t) = v ■ t. 

Coordinate frames. We assign a body-frame to the mobile optical sensor in a way that is 
consistent with standard kinematic models of planar nonholonomic vehicles. Thus, the body- 
frame s-axis is aligned with the direction of motion, and consequently, the image is projected 
onto the body-frame y-axis. We continue to consider Markovian obstacle fields as described in 
Section [2} Thus we consider linear arrays of obstacle slats (line segments) between which it is 
desired to have the mobile sensor move. A frame of world coordinates is assigned such that 
the obstacle slats (line segments) are aligned with the world-frame x-axis, and the direction in 
which the sensor hopes to move is aligned with the world-frame y-axis. 

The mobile sensor coordinates at time t are (x(t),y(t), 9(t)), giving its position and orientation 
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with respect to the world frame. The motion is described by a kinematic model: 



U\ 



V J 



' v cos 9 » 



v sin# 

to 



(3.5) 



/ 



where v is its forward speed (in the direction of the sensor's body-frame x-axis, and uj is the 
turning rate. 

If (x r ,y r ) is the coordinate pair that locates the obstacle point O r in world coordinates, the 
corresponding point on the image (a point along the body-frame y-axis) is 

-sm9(t)(x r - x(t)) + cos 9{t)(y r - y(t)) 



d T {t) 



(3.6) 



1 - cos#(£)(x r - x{t)) - sm9(t)(y r - y(t)) ' 
It is assumed that the optical sensor focal lenth / = 1, here (x(t),y(t), 9(t)) are the position and 
orientation of the sensor frame with respect to world coordinates at time t. A similar relationship 
holds for Ot and d((t). If the mobile sensor proceeds along the straight line that is shown in 
Fig. |4j it will cross a line that passes through Og and is perpendicular to the line of travel before 
it crosses a similar line perpendicular to its path and passing through O r . Note that this line is 
rotated by an amount —9{t) with respect to the line of obstacle slats. As depicted, if the optical 
sensor continues along its straight line path, it will pass between the obstacles that are shown. 
Following the reasoning of Section [l] the time to crossing the line through Og will be de/de, 
provided a straight line motion at constant velocity (say v) is carried out. This assumption of 
straight-line, constant speed motions is crucial in order to interpret dt/dt as the time-to-transit 
the line through Oe. To pursue this assumption in computing and interpreting d? or d r , we treat 9 



as a constant: 9(t) = 9. A straightforward calculation then shows that (3.6) may be more simply 



stated in terms of the world frame initial position (x(Q),y(Q)) and the constant speed v as 



d r (t) 



cos0(y r —y(O)—vs'm0)—sin9(x r —x(O)—vcos9) 
l—cos6(x r —x(0)—vcos6) sin8(yo—y(0)—vsint 

cos9(y r —y(0))—s'm9(x r —x(0)) 
l-cos6(x r -x(O))-sm0(y o -y(O))+vf 



Taking the derivative with respect to t, the time-to-transit the perpendicular line through O r 
given a constant straight line velocity v is 

d = cos g(x r -x(0)) + sin g(y r -y (0))-l _ f 

v 

The interpretation of this formula is that if constant velocity motion is carried out in the 
direction (cos 9, sin 9) given the starting point (x(0),y(0)), then the sensor will transit the line 
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perpendicular to its path and passing through the point (x r ,y r ) when t = (cos9(x r — x(0)) + 
sin 0(y r — 2/(0)) — l)/v. In other words, the time-to-transit from the starting point (x(0),y(0)) 
assuming a straight line velocity (vcos9,v sin9) is 

r r = d r (0)/d r {0) = {cos9(x r -x{0)) + sin9(y r - y(0)) - l)/v. 

These derivations illustrate the relationships between r r and the vehicle kinematic variables. 
It is important to re-emphasize, however, that r r is determined solely from quantities measured 
by the image sensor, and no knowledge of x, y, 9, or v is assumed. Similar considerations apply 
to the left feature point Oe and ti = de(Q) / de(0) . 

It is possible to use inferred time-to-transit quantities to generate steering signals for motion 
control of the mobile point sensor as follows. In the formula for r r , the starting point (x(Q),y(0)) 
may be chosen arbitrarily. In particular, it could be any point along a vehicle trajectory, and hence 
for any trajectory (x(t),y(t),6(t)), we write 

r r (t) = (cos[9(t)}(x r -x(t))+sin[9(t)}(y r -y(t))-l)/v, (3.7) 

which has the interpretation that if the mobile sensor were to proceed at constant speed v in the 
direction (cos 9(t), sin 9(t)) along the path segment 

x(s) \ ( x(t) + vs cos 9(t) \ 

= , 0<s< r r (t) 

y{s) J \ y{t)+vssm9{t) J 

for r r {t) units of time, it would transit the line perpendicular to this straight line path and passing 
through (x r ,y r ). We note that thinking of the transit times r r (£) and Ti(t) as variable quantities 
that depend on the vehicle kinematics in this way allows useful inferences (e.g. is the vehicle 
slowing or accelerating) about motion relative to the associated feature points. 

A. Steering control using image data feedback 

As noted in 0]|,[|2]|, kinematic control laws using sensor measurements referenced to the body 
frame must be understood in light of Brockett's nonstabilizibility theorem. More specifically, no 
configuration (xi,yi,9i) can be asymptotically stabilized by a kinematic control law in which 
the velocity variables v,co in ( |3.5[ ) depend continuously on sensor measurements. This is well 
illustrated by a feedback law based on distance and bearing measurements of the type provided 
by LIDAR or binocular vision. Suppose such a sensor accurately measures the vehicle's distance 
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p and bearing with respect to a feature point — say O r = (x r ,y r ). In 0, it was shown that for 
selected values of a gain parameter A the feedback control law 

v(p) — X(p — 0), and u(p,4>) = p sin (ft — d (3.8) 



will always steer the system (3.5 ) to a point on a circle of prescribed radius d centered at a goal 



point. More specifically, we have the following theorem whose proof is given in [0. 



Theorem 3.1. (7|2]/) Consider the controlled system (3.5) with initial conditions (xo,yo,0o), and 



consider a goal point with coordinates (x r ,y r ). For the control law ^3.8 ), suppose A is in the 



interval < A < d. If the initial configuration of the vehicle body frame (xq, yo,0o) is not directed 
along the unstable branch of p sin — d = 0, then the motion of ( |3.5 ) tends asymptotically to 



(p,0) = (d, 7r/2). The corresponding (x,y,6) trajectory generated by (3.5) tends asymptotically 
to a point on the circle 

(x - x r f + {y- y r f = d 2 

with the body-frame x-axis [in the direction (cos 9, sin 9)] tangent to the circle, pointed in the 
counterclockwise direction. In a completely symmetric fashion, the feedback law v — X(p — d), 
uj = p sin + d produces trajectories tangentially approaching the circle of radius d about the 
goal point in the clockwise direction of tangency. 

Remark 4. Servoing on feature perceptions: One of the great challenges in using body -referenced 
sensing is illustrated by the singular variety psin0 — d = and its apparent movement in 
the world frame as the vehicle moves. The singular variety is a manifestation of Brockett's 
nonstabilizibility theorem, and it specifies a set of configurations from which the goal points 



cannot be reached by the control law (3.8). In (x,y) coordinates, the singular variety can be 
constructed as follows. Draw a circle of radius d around the goal point (x r , y r ). Next draw a line 
from the current vehicle position (x,y) to the goal point. Draw a line that is is tangent to the 
circle and that makes an angle ix — with the previous line as depicted in Fig. [5} This is the stable 
branch of the singular variety. The semi-infinite red line making an angle with the line to the 
goal point is the unstable branch. This unstable branch characterizes the initial configurations 
from which the control law fails to asymptotically approach tangency to the circle. Singular sets 
of this type are characteristic of systems to which Brockett's nonstabilizability theorem applies. 
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p sin cj> — d > 




/? sin (p — d < 



Fig. 5. The singular set (red line segments) associated with the distance/bearing feedback law l |3.8| l. Given the goal point at 
the center of the circle and the current vehicle at the intersection of the two lines, it is the point locus of = p sin <f> — d. 



As the vehicle motion proceeds, the world-frame locus of the singular variety changes — with 
the vehicle's position always being at the intersection of the two branches of the singular locus. 
The singular locus is characteristic of the way that features evolve when rendered in terms of 
world coordinates. Similar feature dynamics are seen in the lines of transit associated with a 
feature point and the time-to-transit parameter r introduced above. As the vehicle moves, these 
lines of transit rotate about the feature point. This apparent motion plays a role in the design 
and analysis of a time-to-transit feedback law that enables passage between the feature points 
Ot and O r . 

In 0, the proof of Theorem |3.1| is carried out in two steps, the first of which specifies a set 



in terms of the body -referenced variables p and <\> that is invariant under the controlled motions. 
The second step is to show that all motions starting in this invariant set tend asymptotically to 
the specified limits. In the following, a control law and its invariant set are specified in terms of 
our image parameters. 



Theorem 3.2. Consider the mobile sensor depicted in Fig. ^and having motion kinematics (3.5). 
Let e > be a small positive constant. The image-referenced configuration set 

di < — e and d r > e (3.9) 
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is invariant under the feedback control law 

v(t) = min(l, T£ + r, 



'/ 



if d r = e or d( = —e 

U (t) ; 

T r — Tg if d r > e and a£ < — e. 

Sketch: Under the assumed model of the obstacle field, we have y r = yt. I.e. the left 
obstacle and the right obstacle, between which we wish to pass, are the same distance from the 
x-axis. We assume that the desired motion is in the direction of the positive y-axis, and hence 
throughout the time over which the control law operates, we have the vehicle's y coordinate 

V < Vr- 

We consider two cases: First, suppose that the vehicle is located at a point (x,y) with xi < 



x < x r and y < y r . If this is the case and (3.9) holds, the heading of the vehicle is such that in 



the absence of steering (i.e. to = 0) it would approach the line segment connecting (x£,yt) and 



(x r , y r ). It can be shown that the set X£ < x < x r ,y < y r , and (3.9 ) is invariant, and that for some 
T > and all t > T, di < — e, d r > e. Thus, the vehicle will be steered by u(t) = T r (t) — Tg(t), 
and it will turn so as to equalize these transit times, and thus it asymptotically approaches the 
line segment between (xe,yg) and (x r ,y r ) in an asymptotically perpendicular direction. 

In the second case, the vehicle position (x, y) is assumed to lie outside the strip xi < x < x r . 
We assume without loss of generality that x > x r . (The case x < xt is treated symmetrically.) 
If the vehicle is such that d r > e, then because r r — Tg < 0, it will turn toward the point (x r , y r ). 
It may enter the strip xe < x < x r , in which case the motion beyond that point is as discussed 
in case one above. If it does not enter the strip, the value of d r will continue to decrease. Once 
d r = e, it will remain at that value until the vehicle has nearly reached the line segment joining 
(xg,y£) and (x r ,y r ). At that point ti ~ r r , and the vehicle approaches the line segment in an 
asymptotically perpendicular direction, and it passes close to the point (x r ,y r ). ■ 

Space does not allow a more complete proof. This will be given elsewhere. 

4. Optical Flow Implementation 

The research described above is supported by an application platform which will be useful 
in understanding issues in our study of animal-inspired flight dynamics. We have developed 
a custom UAV based on the popular quadcopter airframe, equipped with motion sensors, an 
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onboard camera, and a Gumstix Fire SBC (Single Board Computer). Our aim is to demonstrate 
that the real-time, robust measurement necessary for navigation as discussed earlier in the paper is 
possible with these well established technologies together with appropriately selected algorithms 
for real-time optical flow processing. 

Modern optical flow algorithms can be roughly grouped into two families: sparse flow and 
dense flow. In sparse optical flow, only certain points are tracked, and relatively simple models 



such as (1.2) describe the motion of feature points on the image plane. In dense optical flow the 

entire warping transformation is described by the well known intensity flow equation 

dl dl dl 

^-v x + —v y + — = 0. (4.10) 

ox ay ot 

Sparse, i.e. local, methods are both more robust under noise and simpler to implement in 
real-time applications. Like all finite approximations to continuum problems, sparse methods 
can be criticized for not adequately allowing reconstruction of the entire flow field (L3J), but 
for feedback control applications their superiority seems clear. The basic sparse flow algorithm 
that has been tested is derived from OpenCV's Lucas-Kanade implementation. Feature points 
are either found in high texture regions of the scene or are taken at mesh points, and these 
are tracked to give flow fields of adequate density to implement the types of control algorithms 
described in the previous section. Our lab is currently conducting indoor free flight tests together 
with selected tethered outdoor flight tests. 

A. Test flight examples 




Fig. 6. The kinetics can be clearly seen, as the tree stands out compared to its immediate surroundings. 

Figures [6] and [7] show views from an aerial vehicle flying down a narrow sidewalk, bracketed 
by trees on the left and a wall on the right. These images were used in calculating optical flow, 
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Fig. 7. Showing time-to-contact in a close up of the tree. 

without any image preprocessing (aside from the necessary conversion to greyscale). In addition, 
they were not chosen due to their lack of noise, but instead due to the interesting features they 
present. 

In Fig. [6} the proximity of the tree can clearly be inferred from the flow field's discontinuity. 
Objects in the background have minimal flow, but the tree itself has perhaps the highest flow in 
the scene. This demonstrates graphically the fact that it is not necessary to recognize obstacles 
in order to avoid colliding with them while using them as way-point beacons for navigating 
through cluttered environments.. 

Indeed, despite the 3D world, the UAV moves almost along a plane parallel to the ground, 
and so the approaches to calculating r as given earlier in the work are perfectly applicable. Thus 
we calculate that the r for the tree is 0.80 seconds, a calculation born out by the video data, 
which showed the traversal of a line perpendicular to the trees center in approx. 3/4 seconds 
after Fig. [6] was taken. 

Figure [7] shows a close up of the tree in Fig. [6| The vectors have been replaced by a simple 
calculation of r. Note that the overall agreement is quite good, as the standard deviation is only 
^6ms, which on the scale of the UAV's movement is an order of magnitude faster than its 
dynamics. 

5. Conclusions and Further Work 

The paper has provided results on two interrelated but distinct aspects of the problem of 
autonomous vehicle flight control through fields of obstacles. Results of Section II quantify the 
difficulty of constructing collision-free motions in terms of the size and density of the obstacles. 
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The mean length of a collision-free uncontrolled path has been shown to vary inversely with 
the average diameter of the obstacles, but it was shown that the variance of this variable grows 
faster than the square of the mean decreases — making uncontrolled motion very risky. It was also 
shown that for what we termed a quantized Dubins vehicle, that the probability of being able 
to steer a vehicle along a collision-free path was sensitive to the level of steering authority, and 
for a field of obstacles of given average size and density there exists a narrow range of critical 
levels of steering authority below which it is almost impossible to move without colliding and 
above which it is virtually certain that a collision-free path can be followed. In Sections I and 
III, several aspects of optical sensing for motion control were discussed. Recalling the time- 
to-contact parameter r from the perceptual psychology literature, we introduced the concept of 
time-to-transit and indicated how each feature point in a mobile sensor's field of view anchored 
a line of transit that rotated in world coordinates as the sensor moved. Taking this motion 
into account, the parameter r = r(t) was seen to vary along any flight path, and by selecting 
appropriate feature points — preferably kinetic boundaries between obstacles and free space, the 
corresponding variables r(t) were shown to be usable as inputs to a control law. An important 
feature of this type of control is that r was shown to depend only on the sensor image, and it 
required no knowledge of the position and velocity of the sensor/vehicle. 

Further research is needed to develop appropriate families of such control laws that can be 
used to steer through obstacle fields of complexity similar to the models of Section II. It is only 
when complex fields are considered that an understanding of how such optical flow enabled 
control might degrade with respect to obstacle size and density. Because this type of motion 
control appears to be employed in the natural world, it will be a great interest to see in the 
performance characteristics that we expect to develop will be observable in the flight behaviors 
of animals. 
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